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SPECIFICATION 

TITLE OF THE INVENTION 

IMAGE PICKUP APPARATUS AND IMAGE PICKUP METHOD 
BACKGROUND OF THE INVENTION 
[Technical Field] 

The present invention relates to an image pickup apparatus. More particularly, 
the present invention relates to an image pickup apparatus which is applied to a video 
camera using a complementary mosaic color coding CCD. 
[Prior Art] 

A single-board CCD color camera system or the like captures color images by 
using a single solid-state image pickup element such as a CCD image sensor. Such a 
system needs to provide each pixel with a different color filter namely a color coding 
filter on the CCD image sensor as an image pickup element. 

FIG. 1 shows an example of a color array structure used for the complementary 
mosaic color coding filter as an example of a color coding filter . In this figure, Cy 
stands for cyan, Ye for yellow, G for green, and Mg for magenta. There are lines NO 
to N5. The ith line is indicated as Ni (0 < i). Each pixel is expressed as the jth pixel 
(0 < j). According to this convention, Xij means color X for the jth pixel on line Ni. 
For example, G12 means color G (green) for the second pixel on line NI . An example 
in FIG. 3 shows that colors are repeated horizontally at a 2-pixel interval such as Cy, 
Ye, Cy, Ye, and so on. Colors are repeated vertically at a 4-pixel (4-line) interval such 
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as Cy, G, Cy, Mg, and so on. Namely, this example uses a 2-pixel interval for 
horizontal repetition and a 4-line interval for vertical repetition. 

FIG. 2 is a block diagram of a camera signal processing system for a video 
camera apparatus as an image pickup apparatus. This system is provided with a CCD 
image sensor 102 which uses such a complementary mosaic color coding filter as 
illustrated in FIG. 1 or 3. 

In FIG. 2, an image pickup signal from an optical system 101 is sensed in a 
CCD image sensor 102 and is sent to a delay circuit 120 via a front end circuit 103. 
The optical system 101 comprises a camera lens, a mechanical shutter, and the like. 
The CCD image sensor 102 functions as an image pickup element. The front end 
circuit 103 comprises a CDS (correlation double sampling) circuit, a GCA (gain 
control amplifier), an A/D (analog/digital) converter, and the like. The CCD image 
sensor 102 is supplied with a pulse signal read from a timing generator 106. The front 
end circuit 103 is supplied with a sampling pulse, an A/D converter drive pulse, and 
the like from the timing generator 106. The timing generator 106 is supplied with a 
control signal from a system control circuit 107. The delay circuit 120 is also supplied 
with a control signal from the system control circuit 107. An output signal from the 
delay circuit 120 is sent to a Y (brightness) process circuit 140 and a C 
(chromaticness) process circuit 160 via a preprocessing circuit 130. The Y process 
circuit 140 and the C process circuit 160 are also provided with a control signal from 
the system control circuit 107. 
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The system in FIG. 2 reads fields from the interlaced scanning CCD. As shown 
in FIG. 3, the system adds vertically adjacent pixel data to each other on every two 
lines in the CCD. The system reads this data in the form of a signal as illustrated in 
FIG. 4. This operation is called 2-line mixed reading. When the system reads every 
two lines N0+N1, N2+N3, and so on for one field by mixing and adding, the system 
then reads every two lines N1+N2, N3+N4, and so on for the next field by mixing and 
adding. For each pixel to be mixed and added, Cy+G is expressed as Sir, Ye+Mg as 
S2r, Cy+Mg as S lb, and Ye+G as S2b. 

FIG. 5 shows chromaticness signal processing including the C process circuit 
160 in FIG. 2 and peripheral circuits. In FIG. 5, the delay circuit 120 retrieves a 
no-delay signal [HHOD], a 1H delay signal [HH1D], and a 2H delay signal [HH2D] 
using two 1H (one horizontal interval or one line) delay devices 121 and 122. The 
delay circuit then sends these signals to the preprocessing circuit 130. In the 
preprocessing circuit 130, an adder 131 adds the above-mentioned signals [HHOD] to 
[HH2D]. A 1/2 multiplier 132 halves these signals like ([HH0D]+[HH2D])/2 and 
sends the processed signal to the C process circuit 160. The delay circuit sends the 1H 
delay signal [HH1D] unmodified to the C process circuit 160. This signal is used for 
processing. In the C process circuit 160, the 1H delay signal [HH1D] and the signal 
([HH0D]+[HH2D])/2 are sent to the postprocessing circuit 162 via a horizontal spatial 
phase synchronization filter 161. An output is retrieved from the postprocessing 
circuit 162. The horizontal spatial phase synchronization filter 161 uses the tap factor 



(1,0,3)/(3,0,1) for a filter operation. With respect to a signal for lines N2+N3 in FIG. 
4, a filter operation using tap factors (1,0,3) signifies multiplying Cy20+Mg30 by the 
weight 1 and multiplying Cy22+Mg32 by the weight 3. (Additionally, there may be 
normalization through the use of division by a sum of factors.) Such a filter operation 
synchronizes horizontal spatial phases. 

During the chromaticness signal processing in FIG. 5, delay lines of the delay 
circuit 120 synchronize vertical spatial phases for the signal [HH1D] and the signal 
([HH0D]+[HH2D])/2. The horizontal spatial phase synchronization filter 161 
synchronizes a horizontal spatial phase for each of these signals as SI and S2. The 
postprocessing circuit 162 generates a chromaticness signal by processing signals S lr, 
S2r, Sib, and S2b whose vertical and horizontal spatial phases are synchronized. 

The above-mentioned signals [HH1D] and ([HH0D]+[HH2D])/2 are expressed 
as follows with regard to pixel rates for the CCD image sensor 102. 
[HH1D]=N2+N3 

([HH0D]+[HH2D])/2 = ((N0+Nl)+(N4+N5))/2 

A chromaticness signal is created at the CCD pixel rate based on data for six 
lines NO to N5 in a vertical direction. Namely, the following filter operation is 
performed in a vertical direction according to filter tap factors. 
(0,0,2,2,0,0)/(l, 1,0,0,1,1) 

A camera signal processing system according to the conventional field reading 
causes a problem during chromaticness processing. Specifically, given that "fsv" 
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signifies a sampling frequency for the CCD pixel rate in a vertical direction, the 

above-mentioned filtering in the vertical direction causes a problem of generating a 

false color at (1/4) fsv in the spatial frequency band. 

BRIEF SUMMARY OF THE INVENTION 

The present invention has been made in consideration of the foregoing. It is 

therefore an object of the present invention to provide an image pickup apparatus 

which can eliminate a false color at (1/4) fsv in the spatial frequency band during 

chromaticness signal processing. 

For the purpose of solving the above-mentioned problem, an image pickup 
apparatus according to the present invention comprises: an image pickup element 
having a color coding filter; spatial phase synchronization means for synchronizing 
spatial phases in horizontal and vertical directions based on output from each line in 
the image pickup element; and synthesis means for generating a synthesized signal 
based on a signal from the spatial phase synchronization means, wherein this signal 
has a spatial phase synchronized horizontally and vertically. The image pickup 
apparatus according to the present invention processes chromaticness of signals from 
this synthesis means. 

The color coding filter is a complementary mosaic color coding filter . This 
complementary mosaic color coding filter comprises: a first line which is an alternate 
repetition of Cy and Ye; a second line which is an alternate repetition of G and Mg; 
a third line which is an alternate repetition of Cy and Ye; and a fourth line which is an 
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alternate repetition of Mg and G. The complementary mosaic color coding filter is 
based on a repetition of two pixels horizontally by four lines vertically. 

The spatial phase synchronization means generates a signal whose horizontal 
and vertical phases are synchronized. The synthesis means performs the following 
operations based on Cy (cyan), Ye (yellow), G (green), and Mg (magenta) in each 
pixel data for that signal. 

Slr = Cy + G, S2r = Ye + Mg 

Slb = Cy + Mg, S2b = Ye + G 
These operations create new signals Sir, S2r, Sib, and S2b. 

An image pickup apparatus according to the present invention comprises: an 
image pickup element having a color coding filter; spatial phase synchronization 
means for synchronizing horizontal and vertical spatial phases based on output from 
each line in the image pickup element; and synthesis means for generating a 
synthesized signal based on a signal from the spatial phase synchronization means, 
wherein this signal has a spatial phase synchronized horizontally and vertically. The 
image pickup apparatus according to the present invention can eliminate a false signal 
at (1/4) fsv in a chromaticness signal and greatly improve images by processing 
chromaticness of signals from this synthesis means. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 

FIG. 1 shows an example of a complementary mosaic color coding filter on a 
CCD image sensor; 




FIG. 2 is a block diagram showing an example of a camera signal processing 
system for a conventional video camera apparatus; 

FIG. 3 illustrates 2-line mixed reading for the CCD image sensor; 
FIG. 4 illustrates a CCD output signal generated by 2-line mixed reading of the 
CCD image sensor; 

FIG. 5 is a block diagram showing part of a configuration associated with the 
conventional chromaticness signal processing; 

FIG. 6 is a block diagram showing a configuration of a video camera apparatus 
to which an embodiment of the present invention is applied; 

FIG. 7 is a block diagram showing part of a configuration associated with the 
chromaticness signal processing which is a major function for an embodiment of the 
present invention; 

FIG. 8 illustrates frame reading on the CCD image sensor; 

FIG. 9 illustrates a CCD output signal when two field signals are used to read 
all pixels from the CCD image sensor by interlaced scanning; and 

FIG. 10 illustrates a signal after rearrangement according to a pixel array on the 
CCD image sensor with reference to the CCD output signaling FIG. 9. 
DETAILED DESCRIPTION OF THE INVENTION 

Embodiments of an image pickup apparatus according to the present invention 
will be described in detail, with reference to the accompanying drawings. FIG. 6 is a 
block diagram showing a system configuration of a video camera apparatus as the 
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embodiment of an image pickup apparatus according to the present invention. FIG. 
7 is a block diagram showing part of a configuration associated with the chromaticness 
signal processing which is a major function for the embodiment of the present 
invention. 

In FIG. 6, an optical system 1 1 in a camera comprises a lens, a mechanical 
shutter, and the like. A CCD image sensor 12, functioning as an image pickup 
element, picks up and photoelectrically converts an optical image from the optical 
system 11. The CCD image sensor 12 can read all pixels. More specifically, all pixels 
are read on a frame basis in an interlaced scanning CCD. The interlaced scanning and 
frame reading generates a 2-field signal. This signal corresponds to an image which 
is exposed at the same time by the mechanical shutter in the optical system 11. 

An image pickup signal generated from the CCD image sensor 12 is supplied 
to a front end circuit 13. The front end circuit 13 comprises a CDS (correlation double 
sampling) circuit, a GCA (gain control amplifier), an A/D (analog/digital) converter, 
and the like for detecting each pixel's electric charge which is photoelectrically 
converted and stored in the CCD. An output signal from the front end circuit 13 is 
sent to a delay circuit 20 via a memory control circuit 14. Memory 15 is connected to 
the memory control circuit 14. The memory 1 5 is used for rearranging the frame-read 
signal from the CCD image sensor 12 according to spatial array data on the CCD. 

The CCD image sensor 12 is supplied with a read pulse signal from a timing 
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generator 16. The front end circuit 13 is supplied with a sampling pulse, an A/D 
converter drive pulse, and the like from the timing generator 16. The timing generator 
16 is supplied with a control signal from a system control circuit 17. A delay circuit 
120 is also supplied with a control signal from the system control circuit 17. 

The following describes the frame read operation from the CCD image sensor 
12 and the rearrangement in the memory 15 with reference to FIGS. 8 to 10. FIG. 8 
illustrates frame reading on the CCD image sensor 12. This operation reads all pixels 
on all lines NO to N5. Since interlaced scanning is performed every other line, a CCD 
output signal is read as a 2-field signal as shown in FIG. 9. This signal comprises a 
first field Fl for even-numbered lines NO, N2, and N4, and a second field F2 for 
odd-numbered lines Nl, N3, and N5. The memory control circuit 14 controls to write 
or read a signal for two fields Fl and F2 from memory 15. As shown in FIG. 10, this 
signal is arranged for output in the order of spatial arrays on the CCD. The output 
signal from the memory control circuit 14 in FIG. 10 becomes a frame image signal 
which follows the order of lines NO to N5 on the CCD. 

Again in FIG. 6, an output signal from the delay circuit 20 is sent to a Y 
(brightness) process circuit 40 and a C (chromaticness) process circuit 60 via a 
preprocessing circuit 30. A control signal from the system control circuit 17 is also 
supplied to the Y process circuit 40 and the C process circuit 60. 

FIG. 7 shows part of a configuration associated with the chromaticness signal 
processing which is a major function for the embodiment of the present invention. In 
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this FIG. 7, the delay circuit 20 retrieves a no-delay signal [HOD] and 1H delay signal 
[HID] to 5H delay signal [H5D], and sends these signals to a preprocessing circuit 30 
using five 1H (one horizontal interval or one line) delay devices 2 1 to 25 . With regard 
to correspondence among these signals [HOD] to [H5D] and lines NO to N5 in FIG. 10, 
for example, the no-delay signal [HOD] corresponds to line N5. The 1H delay signal 
[HID], the 2H delay signal [H2D], ... and the 5H delay signal [H5D] correspond to the 
line N4, the line N3, ... and the line NO, respectively. 

In the preprocessing circuit 30, the signals [HOD] to [H5D] are added as 
follows. An adder 31 adds signals [HOD] to [HID]. An adder 32 adds signals [H2D] 
to [H3D]. An adder 33 adds signals [H4D] to [H5D]. An adder 34 adds output from 
the adder 3 1 to output from the adder 33. A 1/2 multiplier 35 halves output from the 
adder 34. The preprocessing circuit 30 sends an output signal 
([H0D]+[HlD]+[H4D]+[H5D])/2 from the multiplier 35 and an output signal 
([H2D]+[H3D]) from the adder 32 to a Y process circuit 40. An output signal 
([H2D]+[H3D]) from the adder 32 is to be processed currently. The signals [HOD] to 
[H5D] are sent to a synthesizer circuit 38 via a horizontal spatial phase 
synchronization filter 36 and a vertical spatial phase synchronization filter 37. Signals 
S lr/S2r and S lb/S2b from the synthesizer circuit 38 are sent to a C process circuit 60. 
The horizontal spatial phase synchronization filter 36 uses the following tap factors 
for filter operations. 

(1,0,3)/(3,0,1) 
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The vertical spatial phase synchronization filter 37 uses the following tap factors for 
filter operations. 

(1,0,4,0,3,0)/(0,3,0,4,0,1) 
Synthesis processing in the synthesizer circuit 38 means creating signals S lr, S2r 5 S lb, 
and S2b based on complementary data (Cy, Ye, G, Mg) whose horizontal and vertical 
spatial phases are synchronized. The signals Sir, S2r, Sib, and S2b are expressed as 

follows. 

Slr = Cy + G, S2r = Ye + Mg 

Sib = Cy + Mg, S2b = Ye + G 
The process circuit 60 directly sends signals S lr/S2r and S lb/S2b from the synthesizer 
circuit 38 to a postprocessing circuit 62, thereby bypassing a horizontal spatial phase 
synchronization filter 61. This is because the preprocessing circuit 30 already 
synchronizes horizontal spatial phases, eliminating the need for processing by a 
horizontal spatial phase synchronization filter 61 in the existing C process circuit 60. 

The following describes in more detail operations in the preprocessing circuit 
30 which provides a main function for chromaticness signal processing according to 
this configuration. The horizontal spatial phase synchronization filter 36 and the 
vertical spatial phase synchronization filter 37 synchronize spatial phases as follows. 
For example, a signal is formatted according to a pixel array on the CCD as shown m 
FIG. 10. This example shows that a region Q contains four pixels Cy22, Ye23, Mg32, 
and G33. Point p is a center for these pixels. An interpolation operation is performed 
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to find data (pixel values) for these complementary colors Cy, Ye, Mg, and G instead 
of these pixels Cy22, Ye23, Mg32, and G33. 

In order to explain the example, it is assumed that Vp is a vertical spatial phase 
and Hp is a horizontal spatial phase of the point p for synchronizing the spatial phases. 
Each pixel has a subscript such as say "23" in Ye23. In this example, "2" represents 
a vertical phase and "3" represents a horizontal phase. Each pixel data at the point p 
can be expressed as CyVpHp, YeVpHp 5 MgVpHp, and GVpHp. The example in FIG. 
10 assumes Vp=2.5 and Hp=2.5. As an example, the following describes how to find 
Cy VpHp using an interpolation or filter operation. In this case, the horizontal spatial 
phase synchronization filter 36 is first used to find pixel data CyOHp, Cy2Hp, and 
Cy4Hp having die horizontal phase Hp=2.5. These pixel data can be found by 
performing the following filter operations using tap factors (3, 0, 1). 

CyOHp = 3*Cy02 + 0*Ye03 + l*Cy04 

Cy2Hp = 3*Cy22 + 0*Ye23 + l*Cy24 

Cy4Hp = 3*Cy42 + 0*Ye43 + l*Cy44 
Generally, operations with tap factor 0 are omitted as follows. 

CyOHp = 3*Cy02 + 1 *Cy04 

Cy2Hp = 3*Cy22 + l*Cy24 

Cy4Hp = 3*Cy42 + l*Cy44 
The same applies to the following description. The horizontal spatial phase 
synchronization filter 36 uses the same tap factors (3, 0, 1) for Mg. However, tap 
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factors (1, 0, 3) are used for Ye and G. 

As mentioned above, the horizontal spatial phase synchronization filter 36 
synchronizes horizontal spatial phases in pixel data such as CyOHp, Cy2Hp 5 and 
Cy4Hp. 

This pixel data is sent to the vertical spatial phase synchronization filter 37 which then 
synchronizes vertical spatial phases in the pixel data. 

The following filter operation uses tap factors ( 1 , 0, 4, 0, 3, 0) to find pixel data having 
the vertical phase Vp=2.5 such as CyVpHp. 

CyVpHp = l*CyOHp + 4*Cy2Hp + 3*Cy4Hp 
The tap factors (1, 0, 4, 0, 3, 0) are used for finding YeVpHp. However, the tap 
factors (0, 3, 0, 4, 0, 1) are used for finding MgVpHp and GVpHp. 

As mentioned above, horizontal and vertical spatial phases are synchronized for 
pixel data CyVpHp, YeVpHp, MgVpHp, and GVpHp. The synthesizer circuit 38 then 
performs the following operations using data for these complementary colors Cy, Ye, 
Mg, and G of the pixel data. 

Sir = Cy + G, S2r = Ye + Mg 

Sib = Cy + Mg, S2b = Ye + G 
These operations create signals Sir, S2r, Sib, and S2b. These signals are sent to the 
postprocessing circuit 62 in the C process circuit 60. 

Signals Sir, S2r, Sib, and S2b from the synthesizer circuit 38 are created 
according to an array of 2 x 2 pixels on the CCD. Given that "fsv" signifies a sampling 
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frequency in the vertical direction, no false color occurs at (1/4) fsv. 

The present invention is not limited to the above-mentioned embodiment. 
While the embodiment describes an application to the video camera apparatus, the 
present invention can be applied to other image pickup apparatuses. While the 
embodiment explains signal processing by means of a frame read operation using the 
interlaced scanning CCD, the present invention can be applied to a system which can 
read data for each discrete line in a complementary mosaic color coding CCD. The 
pixel array for complementary mosaic color coding is not limited to the embodiment. 
Other arrays are also available. The present invention can be applied to the use of a 
color coding filter for three primary colors. It is further understood by those skilled 
in the art that various changes and modifications may be made in the invention without 
departing from the spirit and scope thereof. 
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